I, \[**Loubna Sasso**\], confirm that the work presented in this assessment is my own. Where information has been derived from other sources, I confirm that this has been indicated in the work.
date: 02 February, 2022
SQF policy has been the subject of racial profiling over the years, therefore analyzing where SQF polcies are being implemented across different areas of NYC and who they are targeted towards is essential in order to help overcome racial profiling by informing policies for specific precincts to ensure equality. Moreover Research shows that “persons of African and Hispanic descent were stopped more frequently than white people, even after controlling for precinct variability and race-specific estimates of crime participation.” Therefore analyzing where these differences occur in different parts on NYC can be insightful to implement necessary changes.
The databasets used for this analysis comes from NYC open data and New York City Police Department. The New York City Police Department Dataset contains 9545 SQF entries contains entries for and 83 different fields. The dataset includes important information on the various SQF throughout 2020, at different months, and time of day as well as the race and gender of the people being questioned.
The fields that will be used for this analysis include the latitude and longitude coordinates, suspect race description and, month. I will analyse the pattern of sqf on black people over the summer period (june,july,august,spetember). This process will repeated for ‘white people’ if i have time during the 6 hours given, and a comparison between sqf of black vs white people will be made
What patterns off risk (SQF) can be observed in the different precincts in NYC for black people and how might this pattern change in other racial groups?
HYPOTHESIS 1 h0: There are no noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC h1: There are noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC
HYPOTHESIS 2 (moran, getis gerrys) Ho: There are no similarities in spatial autocorrelation of (SQF) policy between black and white people
H1: There are similarities in spatial autocorrelation of (SQF) policy between black and white people
Preprocess the data: - load in data, filter data to black people being questioned and transform it - transform lat and long values to point data, make all points distinct, clip all point to NYC boundary - st_join points to spatial data: group by, summarize and get a density map
Analyzing the data: - Descriptive statistics of all SQF on black people density data: BOX PLOT - Explore clustering of SQF using Point pattern analysis: kerney, ripleys and DBSCAN - Explore different Global methods to analyze levels Spatial autocorrelation of SQF in NYC - Explore different local methods to analyze clustering across communities in NYC
library(spatstat)
## Loading required package: spatstat.data
## Loading required package: spatstat.geom
## spatstat.geom 2.3-0
## Loading required package: spatstat.core
## Loading required package: nlme
## Loading required package: rpart
## spatstat.core 2.3-1
## Loading required package: spatstat.linnet
## spatstat.linnet 2.3-0
##
## spatstat 2.2-0 (nickname: 'That's not important right now')
## For an introduction to spatstat, type 'beginner'
library(here)
## here() starts at C:/Users/loubn/Desktop/old laptop uni/CASA-GIS/GIS FINAL PREP/casa0005-final-exam-LoubnaSasso
library(sp)
library(rgeos)
## rgeos version: 0.5-8, (SVN revision 679)
## GEOS runtime version: 3.9.1-CAPI-1.14.2
## Please note that rgeos will be retired by the end of 2023,
## plan transition to sf functions using GEOS at your earliest convenience.
## GEOS using OverlayNG
## Linking to sp version: 1.4-6
## Polygon checking: TRUE
library(maptools)
## Checking rgeos availability: TRUE
## Please note that 'maptools' will be retired by the end of 2023,
## plan transition at your earliest convenience;
## some functionality will be moved to 'sp'.
library(GISTools)
## Loading required package: RColorBrewer
## Loading required package: MASS
##
## Attaching package: 'MASS'
## The following object is masked from 'package:spatstat.geom':
##
## area
library(tmap)
library(sf)
## Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1
library(geojson)
##
## Attaching package: 'geojson'
## The following object is masked from 'package:graphics':
##
## polygon
library(geojsonio)
## Registered S3 method overwritten by 'geojsonsf':
## method from
## print.geojson geojson
##
## Attaching package: 'geojsonio'
## The following object is masked from 'package:base':
##
## pretty
library(tmaptools)
library(stringr)
library(tidyverse)
## Registered S3 method overwritten by 'cli':
## method from
## print.boxx spatstat.geom
## -- Attaching packages --------------------------------------- tidyverse 1.3.1 --
## v ggplot2 3.3.5 v purrr 0.3.4
## v tibble 3.1.6 v dplyr 1.0.7
## v tidyr 1.1.4 v forcats 0.5.1
## v readr 2.1.0
## -- Conflicts ------------------------------------------ tidyverse_conflicts() --
## x dplyr::collapse() masks nlme::collapse()
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
## x dplyr::select() masks MASS::select()
library(raster)
##
## Attaching package: 'raster'
## The following object is masked from 'package:dplyr':
##
## select
## The following object is masked from 'package:MASS':
##
## select
## The following object is masked from 'package:nlme':
##
## getData
library(fpc)
library(dbscan)
##
## Attaching package: 'dbscan'
## The following object is masked from 'package:fpc':
##
## dbscan
library(ggplot2)
library(OpenStreetMap)
library(janitor)
##
## Attaching package: 'janitor'
## The following object is masked from 'package:raster':
##
## crosstab
## The following objects are masked from 'package:stats':
##
## chisq.test, fisher.test
library(dplyr)
library(spdep)
## Loading required package: spData
## To access larger datasets in this package, install the spDataLarge
## package with: `install.packages('spDataLarge',
## repos='https://nowosad.github.io/drat/', type='source')`
library(plotly)
##
## Attaching package: 'plotly'
## The following object is masked from 'package:raster':
##
## select
## The following object is masked from 'package:ggplot2':
##
## last_plot
## The following object is masked from 'package:MASS':
##
## select
## The following object is masked from 'package:stats':
##
## filter
## The following object is masked from 'package:graphics':
##
## layout
library(rgdal)
## Please note that rgdal will be retired by the end of 2023,
## plan transition to sf/stars/terra functions using GDAL and PROJ
## at your earliest convenience.
##
## rgdal: version: 1.5-27, (SVN revision 1148)
## Geospatial Data Abstraction Library extensions to R successfully loaded
## Loaded GDAL runtime: GDAL 3.2.1, released 2020/12/29
## Path to GDAL shared files: C:/Users/loubn/Documents/R/win-library/4.1/rgdal/gdal
## GDAL binary built with GEOS: TRUE
## Loaded PROJ runtime: Rel. 7.2.1, January 1st, 2021, [PJ_VERSION: 721]
## Path to PROJ shared files: C:/Users/loubn/Documents/R/win-library/4.1/rgdal/proj
## PROJ CDN enabled: FALSE
## Linking to sp version:1.4-6
## To mute warnings of possible GDAL/OSR exportToProj4() degradation,
## use options("rgdal_show_exportToProj4_warnings"="none") before loading sp or rgdal.
## Overwritten PROJ_LIB was C:/Users/loubn/Documents/R/win-library/4.1/rgdal/proj
library(broom)
library(mapview)
library(crosstalk)
library(car)
## Loading required package: carData
##
## Attaching package: 'car'
## The following object is masked from 'package:dplyr':
##
## recode
## The following object is masked from 'package:purrr':
##
## some
## The following object is masked from 'package:spatstat.core':
##
## bc
## The following object is masked from 'package:spatstat.geom':
##
## ellipse
library(fs)
library(tidypredict)
library(corrr)
##
## Attaching package: 'corrr'
## The following object is masked from 'package:raster':
##
## stretch
library(spatialreg)
## Loading required package: Matrix
##
## Attaching package: 'Matrix'
## The following objects are masked from 'package:tidyr':
##
## expand, pack, unpack
##
## Attaching package: 'spatialreg'
## The following objects are masked from 'package:spdep':
##
## aple, aple.mc, aple.plot, as.spam.listw, as_dgRMatrix_listw,
## as_dsCMatrix_I, as_dsCMatrix_IrW, as_dsTMatrix_listw,
## can.be.simmed, cheb_setup, create_WX, do_ldet, eigen_pre_setup,
## eigen_setup, eigenw, errorsarlm, get.ClusterOption,
## get.coresOption, get.mcOption, get.VerboseOption,
## get.ZeroPolicyOption, GMargminImage, GMerrorsar, griffith_sone,
## gstsls, Hausman.test, impacts, intImpacts, invIrM, invIrW,
## Jacobian_W, jacobianSetup, l_max, lagmess, lagsarlm, lextrB,
## lextrS, lextrW, lmSLX, localAple, LU_prepermutate_setup, LU_setup,
## Matrix_J_setup, Matrix_setup, mcdet_setup, MCMCsamp, ME, mom_calc,
## mom_calc_int2, moments_setup, powerWeights, sacsarlm,
## SE_classic_setup, SE_interp_setup, SE_whichMin_setup,
## set.ClusterOption, set.coresOption, set.mcOption,
## set.VerboseOption, set.ZeroPolicyOption, similar.listw, spam_setup,
## spam_update_setup, SpatialFiltering, spautolm, spBreg_err,
## spBreg_lag, spBreg_sac, stsls, subgraph_eigenw, trW
library(spdep)
library(spgwr)
## NOTE: This package does not constitute approval of GWR
## as a method of spatial analysis; see example(gwr)
library(joineR)
## Loading required package: survival
library(units)
## udunits database from C:/Users/loubn/Documents/R/win-library/4.1/units/share/udunits/udunits2.xml
library(ggplot2)
library(xlsx)
library(readxl)
#read in Police data and transform it
police <- st_read(here::here ("Data", "Police Precincts.geojson")) %>%
st_transform(., 32618) #2908
## Reading layer `Police Precincts' from data source
## `C:\Users\loubn\Desktop\old laptop uni\CASA-GIS\GIS FINAL PREP\casa0005-final-exam-LoubnaSasso\Data\Police Precincts.geojson'
## using driver `GeoJSON'
## Simple feature collection with 77 features and 3 fields
## Geometry type: MULTIPOLYGON
## Dimension: XY
## Bounding box: xmin: -74.25559 ymin: 40.49613 xmax: -73.70001 ymax: 40.91553
## Geodetic CRS: WGS 84
#read in sqf data, remove all na values and clean names
sqf <- read_excel(here::here ("Data", "sqf-2020.xlsx"))%>%
clean_names()
for sql: -select fields: suspect_race_description, stop_location_x, stop_location_x, suspect_arrest_offense, month2 -filter: Black, white, white hispanic -drop n/a values -transform
#1) for sqf csv data i want to:
#select columns you want.
sqf <- sqf%>%
dplyr::select(suspect_race_description, stop_location_x, stop_location_y, month2)
# first filter only black and drop na vals in lat and long
sqf_black <- sqf%>%
filter(suspect_race_description == "BLACK")%>%
filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
drop_na(stop_location_x)%>%
drop_na(stop_location_y)
# first filter only white and drop any na vals in lat and long. this will be used later to compare if i have time
sqf_white <- sqf%>%
filter(suspect_race_description == "WHITE HISPANIC" | suspect_race_description == "WHITE")%>%
filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
drop_na(stop_location_x)%>%
drop_na(stop_location_y)
#2) transform csv sqf_black to point data: st_as sf.
sqf_black_sf <- sqf_black%>%
st_as_sf(.,coords = c ( "stop_location_x", "stop_location_y"), crs=2908)%>%
st_transform(.,32618)
#make all points distinct
sqf_black_sf <- sqf_black_sf%>%
distinct(geometry, .keep_all=T)
#3) overlay point data to police districts and clip if required
tmap_mode("view")
## tmap mode set to interactive viewing
tm_shape(police) +
tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_black_sf) +
tm_dots(col = "blue")
#clipping point to xxx map boundary
sqf_black_sf <- sqf_black_sf [police,]
#plot again to see if they are clipped
tmap_mode("view")
## tmap mode set to interactive viewing
tm_shape(police) +
tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_black_sf) +
tm_dots(col = "blue")
lets get an idea of our sqf black data density and distribution
#1) observe areas of highest and lowest density of sqf-black
#could similar densities in adjacent districts mean there is some degree of clustering happening in these areas?
#st_join point data to the police map to allow for continuous data observation
police_sqf_black <- police%>%
st_join(sqf_black_sf)
#count no. of points in each precinct and add area and density
police_sqf_black <- police_sqf_black%>%
add_count(precinct)%>% #counts the no. of points in each precinct (n)
janitor::clean_names()%>%
mutate(area = st_area(.))%>% #get area
mutate (density = n/area)%>% #get density
dplyr::select(density, area, precinct, n) #narror down the colomns you want
#group by borough
police_sqf_black <- police_sqf_black%>%
group_by(precinct)%>%
summarise(density = first(density), precinct = first(precinct), area = first(area),
count = first(n))
#quick choropleth map based on DENSITY
tm_shape(police_sqf_black) +
tm_polygons("density",
style="jenks",
palette="PuOr",
midpoint=NA,
popup.vars=c("precinct", "density"),
title="SQF-Black people Density")
#2) Box plot
#first make sure density col is numeric
police_sqf_black$density_numeric <- as.numeric(police_sqf_black$density)
#boxplot(police_sqf_black$density_numeric)
boxplot(police_sqf_black$density_numeric,
main = "SQF-Black people Density Distribution ",
xlab = "density ",
ylab = "",
col = "orange",
border = "brown",
horizontal = TRUE
)
Highest levels of sqf amongst black people are indicated in dark purple in precincts across Manhattan, west of Bronx and north of Brooklyn. it can also be seen that precincts closer together follow more or less similar sqf of black people density which could indicate similarity in governance in these precincts that are influenced by neighbouring precincts, therefore spatial autocorrelation and clustering occurance.
We can also see the density distribution is positively skewed, meaning that most of the precints in Nyc exhibit lower densities in of sqf amongst black people, however there are still a few areas with high densities. Lets explore further the spatial patterns of these density distribution across the different precinct through point pattern analysis and later morans I
##06 Cluster Analysis: Point pattern Analysis:
ANalysis process: 1. create sp and sf objects 2. Density kernel (which confirms the density map created above) 3. Ripley K (over quadrat) analysis to test clustering present 4. DBSCAN to show where clusters are happening
1.Create sp and ppp objects from the sf point data in order to proceed with ripley k and DBSCAN
#1) now set a window as the precinct nyc boundary
window <- as.owin(police)
plot(window)
#2A) Create a sp object for eviction (sf to sp)
sqf_black_sp <- sqf_black_sf %>%
as(., 'Spatial')
#2B) Create a ppp object
sqf_black_ppp <- ppp(x=sqf_black_sp@coords[,1],
y=sqf_black_sp@coords[,2],
window=window)
#sqf_black_sp@coords[,1]
#3) have a look at the ppp object
sqf_black_ppp %>%
plot(.,pch=16,cex=0.5,
main="SQF Black people")
This confirms the density map produced earlier.
#uses ppp data
sqf_black_ppp %>%
density(., sigma=500) %>%
plot()
We can see from this map that the that the precincts that fall in manhattan , parts of Bronx shows highest densities of SQF concentration, similar to the first density diagram produced. Now lets explore further if the points exhibit clustering using ripleys k.
“To determining whether a pattern exists in the data is to compare the observed distribution of points to an idealised random distribution. If the points do not conform to a random distribution, then it is likely that there is some unobserved factor or factors which could be causing non-random clustering or dispersal to occur” (Adam Dennett and Sam).
Quadrat analysis does not account for different spatial unit sizes such the different sized precincts in NYC, therefore to overcome this limitation, i will jump straight in to a ripleys k test to check if the SQF cases of black people exhibits clustering, This is done by comparing the observed distribution of points with the Poisson random model for a whole range of different distance radii.
#using the ppp data, plot the ripleys K
Ripleys_K <- sqf_black_ppp%>%
Kest(., correction="border") %>%
plot()
Kval_global <- as.data.frame(Kest(sqf_black_ppp, correction = "border"))
Kval_global
## r theo border
## 1 0.00000 0.000000e+00 0.00
## 2 22.79911 1.632998e+03 11901.04
## 3 45.59822 6.531993e+03 54863.40
## 4 68.39734 1.469699e+04 93369.07
## 5 91.19645 2.612797e+04 156446.40
## 6 113.99556 4.082496e+04 228775.26
## 7 136.79467 5.878794e+04 328605.76
## 8 159.59379 8.001692e+04 447167.33
## 9 182.39290 1.045119e+05 560608.39
## 10 205.19201 1.322729e+05 666790.28
## 11 227.99112 1.632998e+05 822915.30
## 12 250.79023 1.975928e+05 967560.83
## 13 273.58935 2.351518e+05 1130273.23
## 14 296.38846 2.759767e+05 1336499.26
## 15 319.18757 3.200677e+05 1509229.26
## 16 341.98668 3.674246e+05 1683968.37
## 17 364.78580 4.180476e+05 1891542.38
## 18 387.58491 4.719365e+05 2116443.25
## 19 410.38402 5.290915e+05 2371491.31
## 20 433.18313 5.895124e+05 2632645.29
## 21 455.98224 6.531993e+05 2862262.06
## 22 478.78136 7.201523e+05 3107248.72
## 23 501.58047 7.903712e+05 3310106.09
## 24 524.37958 8.638561e+05 3625339.91
## 25 547.17869 9.406070e+05 3884404.02
## 26 569.97780 1.020624e+06 4267395.80
## 27 592.77692 1.103907e+06 4551358.65
## 28 615.57603 1.190456e+06 4921733.80
## 29 638.37514 1.280271e+06 5231634.26
## 30 661.17425 1.373352e+06 5552652.60
## 31 683.97337 1.469699e+06 5895016.32
## 32 706.77248 1.569311e+06 6227699.18
## 33 729.57159 1.672190e+06 6614419.70
## 34 752.37070 1.778335e+06 6977882.03
## 35 775.16981 1.887746e+06 7264778.61
## 36 797.96893 2.000423e+06 7575702.49
## 37 820.76804 2.116366e+06 7935309.18
## 38 843.56715 2.235575e+06 8316194.52
## 39 866.36626 2.358050e+06 8677560.44
## 40 889.16538 2.483790e+06 9095402.71
## 41 911.96449 2.612797e+06 9511941.25
## 42 934.76360 2.745070e+06 9997693.18
## 43 957.56271 2.880609e+06 10363077.15
## 44 980.36182 3.019414e+06 10861423.31
## 45 1003.16094 3.161485e+06 11372886.99
## 46 1025.96005 3.306822e+06 11704053.34
## 47 1048.75916 3.455424e+06 12302945.07
## 48 1071.55827 3.607293e+06 12874746.77
## 49 1094.35739 3.762428e+06 13291070.80
## 50 1117.15650 3.920829e+06 13750427.22
## 51 1139.95561 4.082496e+06 14236622.60
## 52 1162.75472 4.247429e+06 14649409.33
## 53 1185.55383 4.415628e+06 15070560.45
## 54 1208.35295 4.587092e+06 15574522.07
## 55 1231.15206 4.761823e+06 16114500.91
## 56 1253.95117 4.939820e+06 16491338.94
## 57 1276.75028 5.121083e+06 16884198.59
## 58 1299.54940 5.305612e+06 17346373.34
## 59 1322.34851 5.493406e+06 18054670.59
## 60 1345.14762 5.684467e+06 18682228.85
## 61 1367.94673 5.878794e+06 19417468.72
## 62 1390.74584 6.076387e+06 19991285.51
## 63 1413.54496 6.277246e+06 20505256.11
## 64 1436.34407 6.481370e+06 20858307.53
## 65 1459.14318 6.688761e+06 21690633.95
## 66 1481.94229 6.899418e+06 22543926.56
## 67 1504.74140 7.113341e+06 22932505.59
## 68 1527.54052 7.330530e+06 23937425.05
## 69 1550.33963 7.550984e+06 24784454.00
## 70 1573.13874 7.774705e+06 25466470.51
## 71 1595.93785 8.001692e+06 26453468.20
## 72 1618.73697 8.231945e+06 26809224.38
## 73 1641.53608 8.465463e+06 27512515.29
## 74 1664.33519 8.702248e+06 28624783.48
## 75 1687.13430 8.942299e+06 29119030.14
## 76 1709.93341 9.185616e+06 30126251.23
## 77 1732.73253 9.432198e+06 31056845.79
## 78 1755.53164 9.682047e+06 31790604.24
## 79 1778.33075 9.935162e+06 32569564.17
## 80 1801.12986 1.019154e+07 33403977.91
## 81 1823.92898 1.045119e+07 33828068.89
## 82 1846.72809 1.071410e+07 34607635.48
## 83 1869.52720 1.098028e+07 36294671.33
## 84 1892.32631 1.124973e+07 37191279.34
## 85 1915.12542 1.152244e+07 37939674.77
## 86 1937.92454 1.179841e+07 39174179.11
## 87 1960.72365 1.207766e+07 39849630.24
## 88 1983.52276 1.236016e+07 41004097.35
## 89 2006.32187 1.264594e+07 42151433.69
## 90 2029.12099 1.293498e+07 42820802.52
## 91 2051.92010 1.322729e+07 43693489.00
## 92 2074.71921 1.352286e+07 45434396.65
## 93 2097.51832 1.382170e+07 46057820.81
## 94 2120.31743 1.412380e+07 47466476.98
## 95 2143.11655 1.442917e+07 49506160.26
## 96 2165.91566 1.473781e+07 50369084.22
## 97 2188.71477 1.504971e+07 51424931.00
## 98 2211.51388 1.536488e+07 54062187.64
## 99 2234.31299 1.568332e+07 55736129.46
## 100 2257.11211 1.600502e+07 58072904.50
## 101 2279.91122 1.632998e+07 60650153.12
## 102 2302.71033 1.665822e+07 62315730.92
## 103 2325.50944 1.698971e+07 63759731.22
## 104 2348.30856 1.732448e+07 64776632.83
## 105 2371.10767 1.766251e+07 67522267.19
## 106 2393.90678 1.800381e+07 68868785.19
## 107 2416.70589 1.834837e+07 69661526.88
## 108 2439.50500 1.869620e+07 70494748.83
## 109 2462.30412 1.904729e+07 72233911.34
## 110 2485.10323 1.940165e+07 73081329.35
## 111 2507.90234 1.975928e+07 74403301.45
## 112 2530.70145 2.012017e+07 75521893.22
## 113 2553.50057 2.048433e+07 76329519.43
## 114 2576.29968 2.085176e+07 77708968.58
## 115 2599.09879 2.122245e+07 78805453.80
## 116 2621.89790 2.159640e+07 79577173.60
## 117 2644.69701 2.197363e+07 80009819.01
## 118 2667.49613 2.235411e+07 81352129.14
## 119 2690.29524 2.273787e+07 81713694.16
## 120 2713.09435 2.312489e+07 81860579.95
## 121 2735.89346 2.351518e+07 82049433.11
## 122 2758.69258 2.390873e+07 84355900.07
## 123 2781.49169 2.430555e+07 85670049.85
## 124 2804.29080 2.470563e+07 86796463.94
## 125 2827.08991 2.510898e+07 86233256.89
## 126 2849.88902 2.551560e+07 85907848.38
## 127 2872.68814 2.592548e+07 87697595.22
## 128 2895.48725 2.633863e+07 88493038.26
## 129 2918.28636 2.675504e+07 90300863.35
## 130 2941.08547 2.717473e+07 90300863.35
## 131 2963.88459 2.759767e+07 91811688.61
## 132 2986.68370 2.802388e+07 104944246.60
## 133 3009.48281 2.845336e+07 106300115.42
## 134 3032.28192 2.888611e+07 108062744.88
## 135 3055.08103 2.932212e+07 109337261.57
## 136 3077.88015 2.976139e+07 108740679.29
## 137 3100.67926 3.020394e+07 110638895.64
## 138 3123.47837 3.064975e+07 NaN
## 139 3146.27748 3.109882e+07 NaN
## 140 3169.07659 3.155116e+07 NaN
## 141 3191.87571 3.200677e+07 NaN
## 142 3214.67482 3.246564e+07 NaN
## 143 3237.47393 3.292778e+07 NaN
## 144 3260.27304 3.339318e+07 NaN
## 145 3283.07216 3.386185e+07 NaN
## 146 3305.87127 3.433379e+07 NaN
## 147 3328.67038 3.480899e+07 NaN
## 148 3351.46949 3.528746e+07 NaN
## 149 3374.26860 3.576920e+07 NaN
## 150 3397.06772 3.625420e+07 NaN
## 151 3419.86683 3.674246e+07 NaN
## 152 3442.66594 3.723400e+07 NaN
## 153 3465.46505 3.772879e+07 NaN
## 154 3488.26417 3.822686e+07 NaN
## 155 3511.06328 3.872819e+07 NaN
## 156 3533.86239 3.923279e+07 NaN
## 157 3556.66150 3.974065e+07 NaN
## 158 3579.46061 4.025178e+07 NaN
## 159 3602.25973 4.076617e+07 NaN
## 160 3625.05884 4.128383e+07 NaN
## 161 3647.85795 4.180476e+07 NaN
## 162 3670.65706 4.232895e+07 NaN
## 163 3693.45618 4.285641e+07 NaN
## 164 3716.25529 4.338713e+07 NaN
## 165 3739.05440 4.392112e+07 NaN
## 166 3761.85351 4.445838e+07 NaN
## 167 3784.65262 4.499890e+07 NaN
## 168 3807.45174 4.554269e+07 NaN
## 169 3830.25085 4.608975e+07 NaN
## 170 3853.04996 4.664007e+07 NaN
## 171 3875.84907 4.719365e+07 NaN
## 172 3898.64819 4.775050e+07 NaN
## 173 3921.44730 4.831062e+07 NaN
## 174 3944.24641 4.887401e+07 NaN
## 175 3967.04552 4.944066e+07 NaN
## 176 3989.84463 5.001057e+07 NaN
## 177 4012.64375 5.058376e+07 NaN
## 178 4035.44286 5.116021e+07 NaN
## 179 4058.24197 5.173992e+07 NaN
## 180 4081.04108 5.232290e+07 NaN
## 181 4103.84019 5.290915e+07 NaN
## 182 4126.63931 5.349866e+07 NaN
## 183 4149.43842 5.409144e+07 NaN
## 184 4172.23753 5.468748e+07 NaN
## 185 4195.03664 5.528679e+07 NaN
## 186 4217.83576 5.588937e+07 NaN
## 187 4240.63487 5.649521e+07 NaN
## 188 4263.43398 5.710432e+07 NaN
## 189 4286.23309 5.771669e+07 NaN
## 190 4309.03220 5.833233e+07 NaN
## 191 4331.83132 5.895124e+07 NaN
## 192 4354.63043 5.957341e+07 NaN
## 193 4377.42954 6.019885e+07 NaN
## 194 4400.22865 6.082756e+07 NaN
## 195 4423.02777 6.145953e+07 NaN
## 196 4445.82688 6.209476e+07 NaN
## 197 4468.62599 6.273326e+07 NaN
## 198 4491.42510 6.337503e+07 NaN
## 199 4514.22421 6.402007e+07 NaN
## 200 4537.02333 6.466837e+07 NaN
## 201 4559.82244 6.531993e+07 NaN
## 202 4582.62155 6.597477e+07 NaN
## 203 4605.42066 6.663286e+07 NaN
## 204 4628.21978 6.729423e+07 NaN
## 205 4651.01889 6.795886e+07 NaN
## 206 4673.81800 6.862676e+07 NaN
## 207 4696.61711 6.929792e+07 NaN
## 208 4719.41622 6.997235e+07 NaN
## 209 4742.21534 7.065004e+07 NaN
## 210 4765.01445 7.133100e+07 NaN
## 211 4787.81356 7.201523e+07 NaN
## 212 4810.61267 7.270272e+07 NaN
## 213 4833.41179 7.339348e+07 NaN
## 214 4856.21090 7.408750e+07 NaN
## 215 4879.01001 7.478479e+07 NaN
## 216 4901.80912 7.548535e+07 NaN
## 217 4924.60823 7.618917e+07 NaN
## 218 4947.40735 7.689626e+07 NaN
## 219 4970.20646 7.760661e+07 NaN
## 220 4993.00557 7.832023e+07 NaN
## 221 5015.80468 7.903712e+07 NaN
## 222 5038.60379 7.975727e+07 NaN
## 223 5061.40291 8.048069e+07 NaN
## 224 5084.20202 8.120737e+07 NaN
## 225 5107.00113 8.193732e+07 NaN
## 226 5129.80024 8.267054e+07 NaN
## 227 5152.59936 8.340702e+07 NaN
## 228 5175.39847 8.414677e+07 NaN
## 229 5198.19758 8.488979e+07 NaN
## 230 5220.99669 8.563607e+07 NaN
## 231 5243.79580 8.638561e+07 NaN
## 232 5266.59492 8.713842e+07 NaN
## 233 5289.39403 8.789450e+07 NaN
## 234 5312.19314 8.865385e+07 NaN
## 235 5334.99225 8.941646e+07 NaN
## 236 5357.79137 9.018233e+07 NaN
## 237 5380.59048 9.095148e+07 NaN
## 238 5403.38959 9.172388e+07 NaN
## 239 5426.18870 9.249956e+07 NaN
## 240 5448.98781 9.327850e+07 NaN
## 241 5471.78693 9.406070e+07 NaN
## 242 5494.58604 9.484618e+07 NaN
## 243 5517.38515 9.563491e+07 NaN
## 244 5540.18426 9.642692e+07 NaN
## 245 5562.98338 9.722219e+07 NaN
## 246 5585.78249 9.802073e+07 NaN
## 247 5608.58160 9.882253e+07 NaN
## 248 5631.38071 9.962760e+07 NaN
## 249 5654.17982 1.004359e+08 NaN
## 250 5676.97894 1.012475e+08 NaN
## 251 5699.77805 1.020624e+08 NaN
## 252 5722.57716 1.028805e+08 NaN
## 253 5745.37627 1.037019e+08 NaN
## 254 5768.17539 1.045266e+08 NaN
## 255 5790.97450 1.053545e+08 NaN
## 256 5813.77361 1.061857e+08 NaN
## 257 5836.57272 1.070202e+08 NaN
## 258 5859.37183 1.078579e+08 NaN
## 259 5882.17095 1.086989e+08 NaN
## 260 5904.97006 1.095432e+08 NaN
## 261 5927.76917 1.103907e+08 NaN
## 262 5950.56828 1.112415e+08 NaN
## 263 5973.36739 1.120955e+08 NaN
## 264 5996.16651 1.129529e+08 NaN
## 265 6018.96562 1.138135e+08 NaN
## 266 6041.76473 1.146773e+08 NaN
## 267 6064.56384 1.155444e+08 NaN
## 268 6087.36296 1.164148e+08 NaN
## 269 6110.16207 1.172885e+08 NaN
## 270 6132.96118 1.181654e+08 NaN
## 271 6155.76029 1.190456e+08 NaN
## 272 6178.55940 1.199290e+08 NaN
## 273 6201.35852 1.208157e+08 NaN
## 274 6224.15763 1.217057e+08 NaN
## 275 6246.95674 1.225990e+08 NaN
## 276 6269.75585 1.234955e+08 NaN
## 277 6292.55497 1.243953e+08 NaN
## 278 6315.35408 1.252983e+08 NaN
## 279 6338.15319 1.262046e+08 NaN
## 280 6360.95230 1.271142e+08 NaN
## 281 6383.75141 1.280271e+08 NaN
## 282 6406.55053 1.289432e+08 NaN
## 283 6429.34964 1.298626e+08 NaN
## 284 6452.14875 1.307852e+08 NaN
## 285 6474.94786 1.317111e+08 NaN
## 286 6497.74698 1.326403e+08 NaN
## 287 6520.54609 1.335727e+08 NaN
## 288 6543.34520 1.345084e+08 NaN
## 289 6566.14431 1.354474e+08 NaN
## 290 6588.94342 1.363897e+08 NaN
## 291 6611.74254 1.373352e+08 NaN
## 292 6634.54165 1.382839e+08 NaN
## 293 6657.34076 1.392360e+08 NaN
## 294 6680.13987 1.401913e+08 NaN
## 295 6702.93898 1.411498e+08 NaN
## 296 6725.73810 1.421117e+08 NaN
## 297 6748.53721 1.430768e+08 NaN
## 298 6771.33632 1.440452e+08 NaN
## 299 6794.13543 1.450168e+08 NaN
## 300 6816.93455 1.459917e+08 NaN
## 301 6839.73366 1.469699e+08 NaN
## 302 6862.53277 1.479513e+08 NaN
## 303 6885.33188 1.489360e+08 NaN
## 304 6908.13099 1.499239e+08 NaN
## 305 6930.93011 1.509152e+08 NaN
## 306 6953.72922 1.519097e+08 NaN
## 307 6976.52833 1.529074e+08 NaN
## 308 6999.32744 1.539085e+08 NaN
## 309 7022.12656 1.549128e+08 NaN
## 310 7044.92567 1.559203e+08 NaN
## 311 7067.72478 1.569311e+08 NaN
## 312 7090.52389 1.579452e+08 NaN
## 313 7113.32300 1.589626e+08 NaN
## 314 7136.12212 1.599832e+08 NaN
## 315 7158.92123 1.610071e+08 NaN
## 316 7181.72034 1.620343e+08 NaN
## 317 7204.51945 1.630647e+08 NaN
## 318 7227.31857 1.640984e+08 NaN
## 319 7250.11768 1.651353e+08 NaN
## 320 7272.91679 1.661755e+08 NaN
## 321 7295.71590 1.672190e+08 NaN
## 322 7318.51501 1.682658e+08 NaN
## 323 7341.31413 1.693158e+08 NaN
## 324 7364.11324 1.703691e+08 NaN
## 325 7386.91235 1.714256e+08 NaN
## 326 7409.71146 1.724855e+08 NaN
## 327 7432.51058 1.735485e+08 NaN
## 328 7455.30969 1.746149e+08 NaN
## 329 7478.10880 1.756845e+08 NaN
## 330 7500.90791 1.767574e+08 NaN
## 331 7523.70702 1.778335e+08 NaN
## 332 7546.50614 1.789129e+08 NaN
## 333 7569.30525 1.799956e+08 NaN
## 334 7592.10436 1.810816e+08 NaN
## 335 7614.90347 1.821708e+08 NaN
## 336 7637.70258 1.832632e+08 NaN
## 337 7660.50170 1.843590e+08 NaN
## 338 7683.30081 1.854580e+08 NaN
## 339 7706.09992 1.865603e+08 NaN
## 340 7728.89903 1.876658e+08 NaN
## 341 7751.69815 1.887746e+08 NaN
## 342 7774.49726 1.898867e+08 NaN
## 343 7797.29637 1.910020e+08 NaN
## 344 7820.09548 1.921206e+08 NaN
## 345 7842.89459 1.932425e+08 NaN
## 346 7865.69371 1.943676e+08 NaN
## 347 7888.49282 1.954960e+08 NaN
## 348 7911.29193 1.966277e+08 NaN
## 349 7934.09104 1.977626e+08 NaN
## 350 7956.89016 1.989008e+08 NaN
## 351 7979.68927 2.000423e+08 NaN
## 352 8002.48838 2.011870e+08 NaN
## 353 8025.28749 2.023350e+08 NaN
## 354 8048.08660 2.034863e+08 NaN
## 355 8070.88572 2.046408e+08 NaN
## 356 8093.68483 2.057986e+08 NaN
## 357 8116.48394 2.069597e+08 NaN
## 358 8139.28305 2.081240e+08 NaN
## 359 8162.08217 2.092916e+08 NaN
## 360 8184.88128 2.104625e+08 NaN
## 361 8207.68039 2.116366e+08 NaN
## 362 8230.47950 2.128140e+08 NaN
## 363 8253.27861 2.139946e+08 NaN
## 364 8276.07773 2.151786e+08 NaN
## 365 8298.87684 2.163657e+08 NaN
## 366 8321.67595 2.175562e+08 NaN
## 367 8344.47506 2.187499e+08 NaN
## 368 8367.27418 2.199469e+08 NaN
## 369 8390.07329 2.211472e+08 NaN
## 370 8412.87240 2.223507e+08 NaN
## 371 8435.67151 2.235575e+08 NaN
## 372 8458.47062 2.247675e+08 NaN
## 373 8481.26974 2.259808e+08 NaN
## 374 8504.06885 2.271974e+08 NaN
## 375 8526.86796 2.284173e+08 NaN
## 376 8549.66707 2.296404e+08 NaN
## 377 8572.46618 2.308668e+08 NaN
## 378 8595.26530 2.320964e+08 NaN
## 379 8618.06441 2.333293e+08 NaN
## 380 8640.86352 2.345655e+08 NaN
## 381 8663.66263 2.358050e+08 NaN
## 382 8686.46175 2.370477e+08 NaN
## 383 8709.26086 2.382937e+08 NaN
## 384 8732.05997 2.395429e+08 NaN
## 385 8754.85908 2.407954e+08 NaN
## 386 8777.65819 2.420512e+08 NaN
## 387 8800.45731 2.433102e+08 NaN
## 388 8823.25642 2.445725e+08 NaN
## 389 8846.05553 2.458381e+08 NaN
## 390 8868.85464 2.471069e+08 NaN
## 391 8891.65376 2.483790e+08 NaN
## 392 8914.45287 2.496544e+08 NaN
## 393 8937.25198 2.509331e+08 NaN
## 394 8960.05109 2.522150e+08 NaN
## 395 8982.85020 2.535001e+08 NaN
## 396 9005.64932 2.547886e+08 NaN
## 397 9028.44843 2.560803e+08 NaN
## 398 9051.24754 2.573752e+08 NaN
## 399 9074.04665 2.586735e+08 NaN
## 400 9096.84577 2.599750e+08 NaN
## 401 9119.64488 2.612797e+08 NaN
## 402 9142.44399 2.625878e+08 NaN
## 403 9165.24310 2.638991e+08 NaN
## 404 9188.04221 2.652136e+08 NaN
## 405 9210.84133 2.665315e+08 NaN
## 406 9233.64044 2.678526e+08 NaN
## 407 9256.43955 2.691769e+08 NaN
## 408 9279.23866 2.705045e+08 NaN
## 409 9302.03778 2.718354e+08 NaN
## 410 9324.83689 2.731696e+08 NaN
## 411 9347.63600 2.745070e+08 NaN
## 412 9370.43511 2.758477e+08 NaN
## 413 9393.23422 2.771917e+08 NaN
## 414 9416.03334 2.785389e+08 NaN
## 415 9438.83245 2.798894e+08 NaN
## 416 9461.63156 2.812431e+08 NaN
## 417 9484.43067 2.826002e+08 NaN
## 418 9507.22978 2.839604e+08 NaN
## 419 9530.02890 2.853240e+08 NaN
## 420 9552.82801 2.866908e+08 NaN
## 421 9575.62712 2.880609e+08 NaN
## 422 9598.42623 2.894343e+08 NaN
## 423 9621.22535 2.908109e+08 NaN
## 424 9644.02446 2.921908e+08 NaN
## 425 9666.82357 2.935739e+08 NaN
## 426 9689.62268 2.949603e+08 NaN
## 427 9712.42179 2.963500e+08 NaN
## 428 9735.22091 2.977430e+08 NaN
## 429 9758.02002 2.991392e+08 NaN
## 430 9780.81913 3.005386e+08 NaN
## 431 9803.61824 3.019414e+08 NaN
## 432 9826.41736 3.033474e+08 NaN
## 433 9849.21647 3.047567e+08 NaN
## 434 9872.01558 3.061692e+08 NaN
## 435 9894.81469 3.075850e+08 NaN
## 436 9917.61380 3.090041e+08 NaN
## 437 9940.41292 3.104265e+08 NaN
## 438 9963.21203 3.118521e+08 NaN
## 439 9986.01114 3.132809e+08 NaN
## 440 10008.81025 3.147131e+08 NaN
## 441 10031.60937 3.161485e+08 NaN
## 442 10054.40848 3.175872e+08 NaN
## 443 10077.20759 3.190291e+08 NaN
## 444 10100.00670 3.204743e+08 NaN
## 445 10122.80581 3.219228e+08 NaN
## 446 10145.60493 3.233745e+08 NaN
## 447 10168.40404 3.248295e+08 NaN
## 448 10191.20315 3.262878e+08 NaN
## 449 10214.00226 3.277493e+08 NaN
## 450 10236.80137 3.292141e+08 NaN
## 451 10259.60049 3.306822e+08 NaN
## 452 10282.39960 3.321535e+08 NaN
## 453 10305.19871 3.336281e+08 NaN
## 454 10327.99782 3.351060e+08 NaN
## 455 10350.79694 3.365871e+08 NaN
## 456 10373.59605 3.380715e+08 NaN
## 457 10396.39516 3.395591e+08 NaN
## 458 10419.19427 3.410501e+08 NaN
## 459 10441.99338 3.425443e+08 NaN
## 460 10464.79250 3.440417e+08 NaN
## 461 10487.59161 3.455424e+08 NaN
## 462 10510.39072 3.470464e+08 NaN
## 463 10533.18983 3.485537e+08 NaN
## 464 10555.98895 3.500642e+08 NaN
## 465 10578.78806 3.515780e+08 NaN
## 466 10601.58717 3.530951e+08 NaN
## 467 10624.38628 3.546154e+08 NaN
## 468 10647.18539 3.561390e+08 NaN
## 469 10669.98451 3.576658e+08 NaN
## 470 10692.78362 3.591959e+08 NaN
## 471 10715.58273 3.607293e+08 NaN
## 472 10738.38184 3.622660e+08 NaN
## 473 10761.18096 3.638059e+08 NaN
## 474 10783.98007 3.653491e+08 NaN
## 475 10806.77918 3.668955e+08 NaN
## 476 10829.57829 3.684453e+08 NaN
## 477 10852.37740 3.699982e+08 NaN
## 478 10875.17652 3.715545e+08 NaN
## 479 10897.97563 3.731140e+08 NaN
## 480 10920.77474 3.746768e+08 NaN
## 481 10943.57385 3.762428e+08 NaN
## 482 10966.37297 3.778121e+08 NaN
## 483 10989.17208 3.793847e+08 NaN
## 484 11011.97119 3.809606e+08 NaN
## 485 11034.77030 3.825397e+08 NaN
## 486 11057.56941 3.841220e+08 NaN
## 487 11080.36853 3.857077e+08 NaN
## 488 11103.16764 3.872966e+08 NaN
## 489 11125.96675 3.888888e+08 NaN
## 490 11148.76586 3.904842e+08 NaN
## 491 11171.56497 3.920829e+08 NaN
## 492 11194.36409 3.936849e+08 NaN
## 493 11217.16320 3.952901e+08 NaN
## 494 11239.96231 3.968986e+08 NaN
## 495 11262.76142 3.985104e+08 NaN
## 496 11285.56054 4.001254e+08 NaN
## 497 11308.35965 4.017437e+08 NaN
## 498 11331.15876 4.033653e+08 NaN
## 499 11353.95787 4.049901e+08 NaN
## 500 11376.75698 4.066182e+08 NaN
## 501 11399.55610 4.082496e+08 NaN
## 502 11422.35521 4.098842e+08 NaN
## 503 11445.15432 4.115221e+08 NaN
## 504 11467.95343 4.131633e+08 NaN
## 505 11490.75255 4.148077e+08 NaN
## 506 11513.55166 4.164554e+08 NaN
## 507 11536.35077 4.181064e+08 NaN
## 508 11559.14988 4.197606e+08 NaN
## 509 11581.94899 4.214181e+08 NaN
## 510 11604.74811 4.230788e+08 NaN
## 511 11627.54722 4.247429e+08 NaN
## 512 11650.34633 4.264102e+08 NaN
## 513 11673.14544 4.280807e+08 NaN
From the graph, we can see that for all distances, SQF locations appear to be clustered up until a distance of approximately 3000. Ripleys k shows us if we have spatial clusters present in our point data, but it does not tell us WHERE clusters are occurring. To discover this we can use DBSAN
Results of the Ripley’s K analysis earlier shows we are getting clustering up to a radius of around 3000m, with the largest bulge in the graph also being at around 3000m radius . Therefore, 3000m is probably a good place to start and we will begin by searching for clusters of at least 4. if this distance isnt suitable, i will explore the elbow plot and use my own judgement to set eps.
Upon attempting the clusters at 3000, it seems to pick up all the points, therefore i have adjuster the eps distance to 900 as per the elbow plot. Moreover, having the cluster threshold at 4, you can see clusters emerge.
#first check the coordinate reference system of the this spatial polygon:
st_geometry(police)
## Geometry set for 77 features
## Geometry type: MULTIPOLYGON
## Dimension: XY
## Bounding box: xmin: 563069.7 ymin: 4483098 xmax: 609762.3 ymax: 4529952
## Projected CRS: WGS 84 / UTM zone 18N
## First 5 geometries:
## MULTIPOLYGON (((580787.3 4504805, 580819.3 4504...
## MULTIPOLYGON (((585412.9 4508493, 585408.9 4508...
## MULTIPOLYGON (((584460.3 4510216, 584702 451008...
## MULTIPOLYGON (((590567.2 4502074, 590552.2 4502...
## MULTIPOLYGON (((584652.4 4502815, 584661.5 4502...
#1.extract the points from the 'spatial points (sp)' data frame
sqf_black_sp_extract <- sqf_black_sp%>% #takes 2A above
coordinates(.)%>%
as.data.frame()
#2. run the DBSCAN using the extracted sp points:
#i choose clustering of eviction to be at 900 and min number of 4 points
DB_sqf_black <- sqf_black_sp_extract %>%
fpc::dbscan(.,eps = 900, MinPts = 4)
#now plot the results
plot(DB_sqf_black, sqf_black_sp_extract , main = "SQF-Black DBSCAN Output", frame = F)
plot(police$geometry, add=T)
#3. lets double check appropriate eps distance with the elbow plot:
sqf_black_sp_extract%>% #takes the point extracted from sp object (1. above)
dbscan::kNNdistplot(.,k=4)
#shows 4000, howveer this seems too large so i'll keep it at 1000 as selected earlier
The outcome of the dbsan confirms that clustering of SQF on black people seems to follow a similarity in the density distribution mapped earlier. Using this map, we can not only confirm there there is clusters of SQF on black people happening, but that the highest clusters are occuring in the same areas mentioned earlier-manhattan, Brooklyn and Bronx.
Could these precincts be influenced by neighboring areas that could perhaps explain their clustering? let explore this spatial autocorrelation using morans I.
According to Waldo R. Tobler, “Everything is related to everything else. But near things are more related than distant things”. Lets investigate precinct-level spatial dependency using localized version of morana’s I in order to see hot spots of where SQF programs on black people are happening.
If they exibit spatial autocorrelation then this would mean that precinct spatial units are influenced by other neighboring precincts and hence may explain potential clusters happening in areas that are geographically close.If this is the case, then similar policies for neighbouring precincts that can be adopted.
Before performing our clustered analysis, we need to generate a spatial weights matrix
#1. calculate the centroids of all polygons in NYC
centroids_police <- police_sqf_black%>%
st_centroid()%>%
st_geometry()
## Warning in st_centroid.sf(.): st_centroid assumes attributes are constant over
## geometries of x
plot(centroids_police)
#2. generate neighbor lists
#queen's case neighbors : doesn't take centroids. taken adjacent neighbors.
NL_queens <- police_sqf_black %>%
poly2nb(., queen=T)
#or k nearest neighbors : take centroids.
NL_KNN <-centroids_police %>%
knearneigh(., k=4)
NL_KNN <- NL_KNN %>%
knn2nb()
#now plot them both: Fist take the NL then the centroid of the spatial object
plot(NL_queens, st_geometry(centroids_police), col="red")
plot(police_sqf_black$geometry, add=T)
plot(NL_KNN, st_geometry(centroids_police), col="blue")
#add a map underneath
plot(police_sqf_black$geometry, add=T)
#3. create a spatial weights matrix object from these weights
# using row standardization 'w'
SWM_queens <- NL_queens %>%
nb2listw(., style="W")
SWM_knn <-NL_KNN %>%
nb2listw(., style="W")
final output map 1:
tmap_mode("plot")
## tmap mode set to plotting
#1. Local morans density
local_moran_density <- police_sqf_black %>%
pull(density_numeric) %>% #column name from police_sqf_black use the density values or count to get local morans
as.vector()%>%
localmoran(., SWM_knn)%>% #spatial weight matrix of knn (generated from before from spatial data)
as_tibble()
slice_head(local_moran_density, n=5) #view the values
## # A tibble: 5 x 5
## Ii E.Ii Var.Ii Z.Ii `Pr(z != E(Ii))`
## <localmrn> <localmrn> <localmrn> <localmrn> <localmrn>
## 1 0.2268173 -0.0010295780 0.019007011 1.6526698 0.09839808
## 2 0.8454488 -0.0154123401 0.280430301 1.6256260 0.10402924
## 3 0.3645525 -0.0079308681 0.145400076 0.9768424 0.32864717
## 4 0.1584960 -0.0005050172 0.009328005 1.6462868 0.09970473
## 5 0.5370529 -0.0104965722 0.191940564 1.2497984 0.21137319
# 2. Copy the I score (column 1) and the z-score sd. (column 4)) back into police_sqf_black data
police_sqf_black <- police_sqf_black %>%
mutate(density_I =as.numeric(local_moran_density$Ii))%>% #local_moran_density is tibble above, specific col =Ii
mutate(density_Iz =as.numeric(local_moran_density$Z.Ii))
#3.lets map them out:
#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)
#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))
tmap_mode("plot")
## tmap mode set to plotting
#plot interactive map
tm_shape(police_sqf_black) +
tm_layout( title = "Local Moran's I-
SQF:Black People")+
tm_layout(frame = FALSE)+
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="Local Moran's I",
legend.hist= TRUE)+
tm_compass() +
tm_scale_bar() +
tm_legend(position=c(1,0.0))+
tm_credits("(c) New York City Police Department", position=c(0.5,0.0))
The limitation of using local morans i alsone is that it is susceptible to shape and sizes of the spatial units therefore in order to overcome this issues, this map should be read along side the dbscan map that shows spatial clusters.
This local morans I map shows rare cases of the highest hot spots of SQF on black people occurring in precincts: 23,24,26,28 and 30. By reading this map along with the dbscan cluster analysis, we can see that these precincts not only exhibit high clusters of SQF cases on black people but also have high influence on each other.
# first filter only white and drop any na vals in lat and long. this will be used later to compare if i have time
sqf_white <- sqf%>%
filter(suspect_race_description == "WHITE HISPANIC" | suspect_race_description == "WHITE")%>%
filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
drop_na(stop_location_x)%>%
drop_na(stop_location_y)
#2) transform csv sqf_white to point data: st_as sf.
sqf_white_sf <- sqf_white%>%
st_as_sf(.,coords = c ( "stop_location_x", "stop_location_y"), crs=2908)%>%
st_transform(.,32618)
#make all points distinct
sqf_white_sf <- sqf_white_sf%>%
distinct(geometry, .keep_all=T)
#3) overlay point data to police districts and clip if required
tmap_mode("view")
## tmap mode set to interactive viewing
tm_shape(police) +
tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_white_sf) +
tm_dots(col = "blue")
#clipping point to xxx map boundary
sqf_white_sf <- sqf_white_sf[police,]
#plot again to see if they are clipped
tmap_mode("view")
## tmap mode set to interactive viewing
tm_shape(police) +
tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_white_sf) +
tm_dots(col = "blue")
Density map of white people
#1) observe areas of highest and lowest density of sqf-white
#st_join point data to the police map to allow for continuous data observation
police_sqf_white <- police%>%
st_join(sqf_white_sf)
#count no. of points in each precinct and add area and density
police_sqf_white <- police_sqf_white%>%
add_count(precinct)%>% #counts the no. of points in each precinct (n)
janitor::clean_names()%>%
mutate(area = st_area(.))%>% #get area
mutate (density = n/area)%>% #get density
dplyr::select(density, area, precinct, n) #narror down the colomns you want
#group by borough
police_sqf_white <- police_sqf_white%>%
group_by(precinct)%>%
summarise(density = first(density), precinct = first(precinct), area = first(area),
count = first(n))
#quick choropleth map based on DENSITY
tm_shape(police_sqf_white) +
tm_polygons("density",
style="jenks",
palette="PuOr",
midpoint=NA,
popup.vars=c("precinct", "density"),
title="SQF-White people Density")
we can see some slight differences when compared to density of black people, however they both show high density in Manhattan. Lets explore the clusters of SQF white people in order to make further comparisons.
Cluster analysis: White people
ANalysis process: 1. create sp and sf objects 2. Density kernel (which confirms the density map created above) 3. Ripley K (over quadrat) analysis to test clustering present 4. DBSCAN to show where clusters are happening
1.Create sp and ppp objects from the sf point data in order to proceed with ripley k and DBSCAN
#1) now set a window as the precinct nyc boundary
window <- as.owin(police)
plot(window)
#2A) Create a sp object for eviction (sf to sp)
sqf_white_sp <- sqf_white_sf %>%
as(., 'Spatial')
#2B) Create a ppp object
sqf_white_ppp <- ppp(x=sqf_white_sp@coords[,1],
y=sqf_white_sp@coords[,2],
window=window)
#sqf_black_sp@coords[,1]
#3) have a look at the ppp object
sqf_white_ppp %>%
plot(.,pch=16,cex=0.5,
main="SQF white people")
This confirms the density map produced earlier.
#uses ppp data
sqf_white_ppp %>%
density(., sigma=500) %>%
plot()
We can see from this map that the similar to SQF black people cases, the precincts that fall in manhattan and part of Bronx also shows highest densities of SQF concentration for white people
#using the ppp data, plot the ripleys K
Ripleys_K <- sqf_white_ppp%>%
Kest(., correction="border") %>%
plot()
Kval_global <- as.data.frame(Kest(sqf_black_ppp, correction = "border"))
Kval_global
## r theo border
## 1 0.00000 0.000000e+00 0.00
## 2 22.79911 1.632998e+03 11901.04
## 3 45.59822 6.531993e+03 54863.40
## 4 68.39734 1.469699e+04 93369.07
## 5 91.19645 2.612797e+04 156446.40
## 6 113.99556 4.082496e+04 228775.26
## 7 136.79467 5.878794e+04 328605.76
## 8 159.59379 8.001692e+04 447167.33
## 9 182.39290 1.045119e+05 560608.39
## 10 205.19201 1.322729e+05 666790.28
## 11 227.99112 1.632998e+05 822915.30
## 12 250.79023 1.975928e+05 967560.83
## 13 273.58935 2.351518e+05 1130273.23
## 14 296.38846 2.759767e+05 1336499.26
## 15 319.18757 3.200677e+05 1509229.26
## 16 341.98668 3.674246e+05 1683968.37
## 17 364.78580 4.180476e+05 1891542.38
## 18 387.58491 4.719365e+05 2116443.25
## 19 410.38402 5.290915e+05 2371491.31
## 20 433.18313 5.895124e+05 2632645.29
## 21 455.98224 6.531993e+05 2862262.06
## 22 478.78136 7.201523e+05 3107248.72
## 23 501.58047 7.903712e+05 3310106.09
## 24 524.37958 8.638561e+05 3625339.91
## 25 547.17869 9.406070e+05 3884404.02
## 26 569.97780 1.020624e+06 4267395.80
## 27 592.77692 1.103907e+06 4551358.65
## 28 615.57603 1.190456e+06 4921733.80
## 29 638.37514 1.280271e+06 5231634.26
## 30 661.17425 1.373352e+06 5552652.60
## 31 683.97337 1.469699e+06 5895016.32
## 32 706.77248 1.569311e+06 6227699.18
## 33 729.57159 1.672190e+06 6614419.70
## 34 752.37070 1.778335e+06 6977882.03
## 35 775.16981 1.887746e+06 7264778.61
## 36 797.96893 2.000423e+06 7575702.49
## 37 820.76804 2.116366e+06 7935309.18
## 38 843.56715 2.235575e+06 8316194.52
## 39 866.36626 2.358050e+06 8677560.44
## 40 889.16538 2.483790e+06 9095402.71
## 41 911.96449 2.612797e+06 9511941.25
## 42 934.76360 2.745070e+06 9997693.18
## 43 957.56271 2.880609e+06 10363077.15
## 44 980.36182 3.019414e+06 10861423.31
## 45 1003.16094 3.161485e+06 11372886.99
## 46 1025.96005 3.306822e+06 11704053.34
## 47 1048.75916 3.455424e+06 12302945.07
## 48 1071.55827 3.607293e+06 12874746.77
## 49 1094.35739 3.762428e+06 13291070.80
## 50 1117.15650 3.920829e+06 13750427.22
## 51 1139.95561 4.082496e+06 14236622.60
## 52 1162.75472 4.247429e+06 14649409.33
## 53 1185.55383 4.415628e+06 15070560.45
## 54 1208.35295 4.587092e+06 15574522.07
## 55 1231.15206 4.761823e+06 16114500.91
## 56 1253.95117 4.939820e+06 16491338.94
## 57 1276.75028 5.121083e+06 16884198.59
## 58 1299.54940 5.305612e+06 17346373.34
## 59 1322.34851 5.493406e+06 18054670.59
## 60 1345.14762 5.684467e+06 18682228.85
## 61 1367.94673 5.878794e+06 19417468.72
## 62 1390.74584 6.076387e+06 19991285.51
## 63 1413.54496 6.277246e+06 20505256.11
## 64 1436.34407 6.481370e+06 20858307.53
## 65 1459.14318 6.688761e+06 21690633.95
## 66 1481.94229 6.899418e+06 22543926.56
## 67 1504.74140 7.113341e+06 22932505.59
## 68 1527.54052 7.330530e+06 23937425.05
## 69 1550.33963 7.550984e+06 24784454.00
## 70 1573.13874 7.774705e+06 25466470.51
## 71 1595.93785 8.001692e+06 26453468.20
## 72 1618.73697 8.231945e+06 26809224.38
## 73 1641.53608 8.465463e+06 27512515.29
## 74 1664.33519 8.702248e+06 28624783.48
## 75 1687.13430 8.942299e+06 29119030.14
## 76 1709.93341 9.185616e+06 30126251.23
## 77 1732.73253 9.432198e+06 31056845.79
## 78 1755.53164 9.682047e+06 31790604.24
## 79 1778.33075 9.935162e+06 32569564.17
## 80 1801.12986 1.019154e+07 33403977.91
## 81 1823.92898 1.045119e+07 33828068.89
## 82 1846.72809 1.071410e+07 34607635.48
## 83 1869.52720 1.098028e+07 36294671.33
## 84 1892.32631 1.124973e+07 37191279.34
## 85 1915.12542 1.152244e+07 37939674.77
## 86 1937.92454 1.179841e+07 39174179.11
## 87 1960.72365 1.207766e+07 39849630.24
## 88 1983.52276 1.236016e+07 41004097.35
## 89 2006.32187 1.264594e+07 42151433.69
## 90 2029.12099 1.293498e+07 42820802.52
## 91 2051.92010 1.322729e+07 43693489.00
## 92 2074.71921 1.352286e+07 45434396.65
## 93 2097.51832 1.382170e+07 46057820.81
## 94 2120.31743 1.412380e+07 47466476.98
## 95 2143.11655 1.442917e+07 49506160.26
## 96 2165.91566 1.473781e+07 50369084.22
## 97 2188.71477 1.504971e+07 51424931.00
## 98 2211.51388 1.536488e+07 54062187.64
## 99 2234.31299 1.568332e+07 55736129.46
## 100 2257.11211 1.600502e+07 58072904.50
## 101 2279.91122 1.632998e+07 60650153.12
## 102 2302.71033 1.665822e+07 62315730.92
## 103 2325.50944 1.698971e+07 63759731.22
## 104 2348.30856 1.732448e+07 64776632.83
## 105 2371.10767 1.766251e+07 67522267.19
## 106 2393.90678 1.800381e+07 68868785.19
## 107 2416.70589 1.834837e+07 69661526.88
## 108 2439.50500 1.869620e+07 70494748.83
## 109 2462.30412 1.904729e+07 72233911.34
## 110 2485.10323 1.940165e+07 73081329.35
## 111 2507.90234 1.975928e+07 74403301.45
## 112 2530.70145 2.012017e+07 75521893.22
## 113 2553.50057 2.048433e+07 76329519.43
## 114 2576.29968 2.085176e+07 77708968.58
## 115 2599.09879 2.122245e+07 78805453.80
## 116 2621.89790 2.159640e+07 79577173.60
## 117 2644.69701 2.197363e+07 80009819.01
## 118 2667.49613 2.235411e+07 81352129.14
## 119 2690.29524 2.273787e+07 81713694.16
## 120 2713.09435 2.312489e+07 81860579.95
## 121 2735.89346 2.351518e+07 82049433.11
## 122 2758.69258 2.390873e+07 84355900.07
## 123 2781.49169 2.430555e+07 85670049.85
## 124 2804.29080 2.470563e+07 86796463.94
## 125 2827.08991 2.510898e+07 86233256.89
## 126 2849.88902 2.551560e+07 85907848.38
## 127 2872.68814 2.592548e+07 87697595.22
## 128 2895.48725 2.633863e+07 88493038.26
## 129 2918.28636 2.675504e+07 90300863.35
## 130 2941.08547 2.717473e+07 90300863.35
## 131 2963.88459 2.759767e+07 91811688.61
## 132 2986.68370 2.802388e+07 104944246.60
## 133 3009.48281 2.845336e+07 106300115.42
## 134 3032.28192 2.888611e+07 108062744.88
## 135 3055.08103 2.932212e+07 109337261.57
## 136 3077.88015 2.976139e+07 108740679.29
## 137 3100.67926 3.020394e+07 110638895.64
## 138 3123.47837 3.064975e+07 NaN
## 139 3146.27748 3.109882e+07 NaN
## 140 3169.07659 3.155116e+07 NaN
## 141 3191.87571 3.200677e+07 NaN
## 142 3214.67482 3.246564e+07 NaN
## 143 3237.47393 3.292778e+07 NaN
## 144 3260.27304 3.339318e+07 NaN
## 145 3283.07216 3.386185e+07 NaN
## 146 3305.87127 3.433379e+07 NaN
## 147 3328.67038 3.480899e+07 NaN
## 148 3351.46949 3.528746e+07 NaN
## 149 3374.26860 3.576920e+07 NaN
## 150 3397.06772 3.625420e+07 NaN
## 151 3419.86683 3.674246e+07 NaN
## 152 3442.66594 3.723400e+07 NaN
## 153 3465.46505 3.772879e+07 NaN
## 154 3488.26417 3.822686e+07 NaN
## 155 3511.06328 3.872819e+07 NaN
## 156 3533.86239 3.923279e+07 NaN
## 157 3556.66150 3.974065e+07 NaN
## 158 3579.46061 4.025178e+07 NaN
## 159 3602.25973 4.076617e+07 NaN
## 160 3625.05884 4.128383e+07 NaN
## 161 3647.85795 4.180476e+07 NaN
## 162 3670.65706 4.232895e+07 NaN
## 163 3693.45618 4.285641e+07 NaN
## 164 3716.25529 4.338713e+07 NaN
## 165 3739.05440 4.392112e+07 NaN
## 166 3761.85351 4.445838e+07 NaN
## 167 3784.65262 4.499890e+07 NaN
## 168 3807.45174 4.554269e+07 NaN
## 169 3830.25085 4.608975e+07 NaN
## 170 3853.04996 4.664007e+07 NaN
## 171 3875.84907 4.719365e+07 NaN
## 172 3898.64819 4.775050e+07 NaN
## 173 3921.44730 4.831062e+07 NaN
## 174 3944.24641 4.887401e+07 NaN
## 175 3967.04552 4.944066e+07 NaN
## 176 3989.84463 5.001057e+07 NaN
## 177 4012.64375 5.058376e+07 NaN
## 178 4035.44286 5.116021e+07 NaN
## 179 4058.24197 5.173992e+07 NaN
## 180 4081.04108 5.232290e+07 NaN
## 181 4103.84019 5.290915e+07 NaN
## 182 4126.63931 5.349866e+07 NaN
## 183 4149.43842 5.409144e+07 NaN
## 184 4172.23753 5.468748e+07 NaN
## 185 4195.03664 5.528679e+07 NaN
## 186 4217.83576 5.588937e+07 NaN
## 187 4240.63487 5.649521e+07 NaN
## 188 4263.43398 5.710432e+07 NaN
## 189 4286.23309 5.771669e+07 NaN
## 190 4309.03220 5.833233e+07 NaN
## 191 4331.83132 5.895124e+07 NaN
## 192 4354.63043 5.957341e+07 NaN
## 193 4377.42954 6.019885e+07 NaN
## 194 4400.22865 6.082756e+07 NaN
## 195 4423.02777 6.145953e+07 NaN
## 196 4445.82688 6.209476e+07 NaN
## 197 4468.62599 6.273326e+07 NaN
## 198 4491.42510 6.337503e+07 NaN
## 199 4514.22421 6.402007e+07 NaN
## 200 4537.02333 6.466837e+07 NaN
## 201 4559.82244 6.531993e+07 NaN
## 202 4582.62155 6.597477e+07 NaN
## 203 4605.42066 6.663286e+07 NaN
## 204 4628.21978 6.729423e+07 NaN
## 205 4651.01889 6.795886e+07 NaN
## 206 4673.81800 6.862676e+07 NaN
## 207 4696.61711 6.929792e+07 NaN
## 208 4719.41622 6.997235e+07 NaN
## 209 4742.21534 7.065004e+07 NaN
## 210 4765.01445 7.133100e+07 NaN
## 211 4787.81356 7.201523e+07 NaN
## 212 4810.61267 7.270272e+07 NaN
## 213 4833.41179 7.339348e+07 NaN
## 214 4856.21090 7.408750e+07 NaN
## 215 4879.01001 7.478479e+07 NaN
## 216 4901.80912 7.548535e+07 NaN
## 217 4924.60823 7.618917e+07 NaN
## 218 4947.40735 7.689626e+07 NaN
## 219 4970.20646 7.760661e+07 NaN
## 220 4993.00557 7.832023e+07 NaN
## 221 5015.80468 7.903712e+07 NaN
## 222 5038.60379 7.975727e+07 NaN
## 223 5061.40291 8.048069e+07 NaN
## 224 5084.20202 8.120737e+07 NaN
## 225 5107.00113 8.193732e+07 NaN
## 226 5129.80024 8.267054e+07 NaN
## 227 5152.59936 8.340702e+07 NaN
## 228 5175.39847 8.414677e+07 NaN
## 229 5198.19758 8.488979e+07 NaN
## 230 5220.99669 8.563607e+07 NaN
## 231 5243.79580 8.638561e+07 NaN
## 232 5266.59492 8.713842e+07 NaN
## 233 5289.39403 8.789450e+07 NaN
## 234 5312.19314 8.865385e+07 NaN
## 235 5334.99225 8.941646e+07 NaN
## 236 5357.79137 9.018233e+07 NaN
## 237 5380.59048 9.095148e+07 NaN
## 238 5403.38959 9.172388e+07 NaN
## 239 5426.18870 9.249956e+07 NaN
## 240 5448.98781 9.327850e+07 NaN
## 241 5471.78693 9.406070e+07 NaN
## 242 5494.58604 9.484618e+07 NaN
## 243 5517.38515 9.563491e+07 NaN
## 244 5540.18426 9.642692e+07 NaN
## 245 5562.98338 9.722219e+07 NaN
## 246 5585.78249 9.802073e+07 NaN
## 247 5608.58160 9.882253e+07 NaN
## 248 5631.38071 9.962760e+07 NaN
## 249 5654.17982 1.004359e+08 NaN
## 250 5676.97894 1.012475e+08 NaN
## 251 5699.77805 1.020624e+08 NaN
## 252 5722.57716 1.028805e+08 NaN
## 253 5745.37627 1.037019e+08 NaN
## 254 5768.17539 1.045266e+08 NaN
## 255 5790.97450 1.053545e+08 NaN
## 256 5813.77361 1.061857e+08 NaN
## 257 5836.57272 1.070202e+08 NaN
## 258 5859.37183 1.078579e+08 NaN
## 259 5882.17095 1.086989e+08 NaN
## 260 5904.97006 1.095432e+08 NaN
## 261 5927.76917 1.103907e+08 NaN
## 262 5950.56828 1.112415e+08 NaN
## 263 5973.36739 1.120955e+08 NaN
## 264 5996.16651 1.129529e+08 NaN
## 265 6018.96562 1.138135e+08 NaN
## 266 6041.76473 1.146773e+08 NaN
## 267 6064.56384 1.155444e+08 NaN
## 268 6087.36296 1.164148e+08 NaN
## 269 6110.16207 1.172885e+08 NaN
## 270 6132.96118 1.181654e+08 NaN
## 271 6155.76029 1.190456e+08 NaN
## 272 6178.55940 1.199290e+08 NaN
## 273 6201.35852 1.208157e+08 NaN
## 274 6224.15763 1.217057e+08 NaN
## 275 6246.95674 1.225990e+08 NaN
## 276 6269.75585 1.234955e+08 NaN
## 277 6292.55497 1.243953e+08 NaN
## 278 6315.35408 1.252983e+08 NaN
## 279 6338.15319 1.262046e+08 NaN
## 280 6360.95230 1.271142e+08 NaN
## 281 6383.75141 1.280271e+08 NaN
## 282 6406.55053 1.289432e+08 NaN
## 283 6429.34964 1.298626e+08 NaN
## 284 6452.14875 1.307852e+08 NaN
## 285 6474.94786 1.317111e+08 NaN
## 286 6497.74698 1.326403e+08 NaN
## 287 6520.54609 1.335727e+08 NaN
## 288 6543.34520 1.345084e+08 NaN
## 289 6566.14431 1.354474e+08 NaN
## 290 6588.94342 1.363897e+08 NaN
## 291 6611.74254 1.373352e+08 NaN
## 292 6634.54165 1.382839e+08 NaN
## 293 6657.34076 1.392360e+08 NaN
## 294 6680.13987 1.401913e+08 NaN
## 295 6702.93898 1.411498e+08 NaN
## 296 6725.73810 1.421117e+08 NaN
## 297 6748.53721 1.430768e+08 NaN
## 298 6771.33632 1.440452e+08 NaN
## 299 6794.13543 1.450168e+08 NaN
## 300 6816.93455 1.459917e+08 NaN
## 301 6839.73366 1.469699e+08 NaN
## 302 6862.53277 1.479513e+08 NaN
## 303 6885.33188 1.489360e+08 NaN
## 304 6908.13099 1.499239e+08 NaN
## 305 6930.93011 1.509152e+08 NaN
## 306 6953.72922 1.519097e+08 NaN
## 307 6976.52833 1.529074e+08 NaN
## 308 6999.32744 1.539085e+08 NaN
## 309 7022.12656 1.549128e+08 NaN
## 310 7044.92567 1.559203e+08 NaN
## 311 7067.72478 1.569311e+08 NaN
## 312 7090.52389 1.579452e+08 NaN
## 313 7113.32300 1.589626e+08 NaN
## 314 7136.12212 1.599832e+08 NaN
## 315 7158.92123 1.610071e+08 NaN
## 316 7181.72034 1.620343e+08 NaN
## 317 7204.51945 1.630647e+08 NaN
## 318 7227.31857 1.640984e+08 NaN
## 319 7250.11768 1.651353e+08 NaN
## 320 7272.91679 1.661755e+08 NaN
## 321 7295.71590 1.672190e+08 NaN
## 322 7318.51501 1.682658e+08 NaN
## 323 7341.31413 1.693158e+08 NaN
## 324 7364.11324 1.703691e+08 NaN
## 325 7386.91235 1.714256e+08 NaN
## 326 7409.71146 1.724855e+08 NaN
## 327 7432.51058 1.735485e+08 NaN
## 328 7455.30969 1.746149e+08 NaN
## 329 7478.10880 1.756845e+08 NaN
## 330 7500.90791 1.767574e+08 NaN
## 331 7523.70702 1.778335e+08 NaN
## 332 7546.50614 1.789129e+08 NaN
## 333 7569.30525 1.799956e+08 NaN
## 334 7592.10436 1.810816e+08 NaN
## 335 7614.90347 1.821708e+08 NaN
## 336 7637.70258 1.832632e+08 NaN
## 337 7660.50170 1.843590e+08 NaN
## 338 7683.30081 1.854580e+08 NaN
## 339 7706.09992 1.865603e+08 NaN
## 340 7728.89903 1.876658e+08 NaN
## 341 7751.69815 1.887746e+08 NaN
## 342 7774.49726 1.898867e+08 NaN
## 343 7797.29637 1.910020e+08 NaN
## 344 7820.09548 1.921206e+08 NaN
## 345 7842.89459 1.932425e+08 NaN
## 346 7865.69371 1.943676e+08 NaN
## 347 7888.49282 1.954960e+08 NaN
## 348 7911.29193 1.966277e+08 NaN
## 349 7934.09104 1.977626e+08 NaN
## 350 7956.89016 1.989008e+08 NaN
## 351 7979.68927 2.000423e+08 NaN
## 352 8002.48838 2.011870e+08 NaN
## 353 8025.28749 2.023350e+08 NaN
## 354 8048.08660 2.034863e+08 NaN
## 355 8070.88572 2.046408e+08 NaN
## 356 8093.68483 2.057986e+08 NaN
## 357 8116.48394 2.069597e+08 NaN
## 358 8139.28305 2.081240e+08 NaN
## 359 8162.08217 2.092916e+08 NaN
## 360 8184.88128 2.104625e+08 NaN
## 361 8207.68039 2.116366e+08 NaN
## 362 8230.47950 2.128140e+08 NaN
## 363 8253.27861 2.139946e+08 NaN
## 364 8276.07773 2.151786e+08 NaN
## 365 8298.87684 2.163657e+08 NaN
## 366 8321.67595 2.175562e+08 NaN
## 367 8344.47506 2.187499e+08 NaN
## 368 8367.27418 2.199469e+08 NaN
## 369 8390.07329 2.211472e+08 NaN
## 370 8412.87240 2.223507e+08 NaN
## 371 8435.67151 2.235575e+08 NaN
## 372 8458.47062 2.247675e+08 NaN
## 373 8481.26974 2.259808e+08 NaN
## 374 8504.06885 2.271974e+08 NaN
## 375 8526.86796 2.284173e+08 NaN
## 376 8549.66707 2.296404e+08 NaN
## 377 8572.46618 2.308668e+08 NaN
## 378 8595.26530 2.320964e+08 NaN
## 379 8618.06441 2.333293e+08 NaN
## 380 8640.86352 2.345655e+08 NaN
## 381 8663.66263 2.358050e+08 NaN
## 382 8686.46175 2.370477e+08 NaN
## 383 8709.26086 2.382937e+08 NaN
## 384 8732.05997 2.395429e+08 NaN
## 385 8754.85908 2.407954e+08 NaN
## 386 8777.65819 2.420512e+08 NaN
## 387 8800.45731 2.433102e+08 NaN
## 388 8823.25642 2.445725e+08 NaN
## 389 8846.05553 2.458381e+08 NaN
## 390 8868.85464 2.471069e+08 NaN
## 391 8891.65376 2.483790e+08 NaN
## 392 8914.45287 2.496544e+08 NaN
## 393 8937.25198 2.509331e+08 NaN
## 394 8960.05109 2.522150e+08 NaN
## 395 8982.85020 2.535001e+08 NaN
## 396 9005.64932 2.547886e+08 NaN
## 397 9028.44843 2.560803e+08 NaN
## 398 9051.24754 2.573752e+08 NaN
## 399 9074.04665 2.586735e+08 NaN
## 400 9096.84577 2.599750e+08 NaN
## 401 9119.64488 2.612797e+08 NaN
## 402 9142.44399 2.625878e+08 NaN
## 403 9165.24310 2.638991e+08 NaN
## 404 9188.04221 2.652136e+08 NaN
## 405 9210.84133 2.665315e+08 NaN
## 406 9233.64044 2.678526e+08 NaN
## 407 9256.43955 2.691769e+08 NaN
## 408 9279.23866 2.705045e+08 NaN
## 409 9302.03778 2.718354e+08 NaN
## 410 9324.83689 2.731696e+08 NaN
## 411 9347.63600 2.745070e+08 NaN
## 412 9370.43511 2.758477e+08 NaN
## 413 9393.23422 2.771917e+08 NaN
## 414 9416.03334 2.785389e+08 NaN
## 415 9438.83245 2.798894e+08 NaN
## 416 9461.63156 2.812431e+08 NaN
## 417 9484.43067 2.826002e+08 NaN
## 418 9507.22978 2.839604e+08 NaN
## 419 9530.02890 2.853240e+08 NaN
## 420 9552.82801 2.866908e+08 NaN
## 421 9575.62712 2.880609e+08 NaN
## 422 9598.42623 2.894343e+08 NaN
## 423 9621.22535 2.908109e+08 NaN
## 424 9644.02446 2.921908e+08 NaN
## 425 9666.82357 2.935739e+08 NaN
## 426 9689.62268 2.949603e+08 NaN
## 427 9712.42179 2.963500e+08 NaN
## 428 9735.22091 2.977430e+08 NaN
## 429 9758.02002 2.991392e+08 NaN
## 430 9780.81913 3.005386e+08 NaN
## 431 9803.61824 3.019414e+08 NaN
## 432 9826.41736 3.033474e+08 NaN
## 433 9849.21647 3.047567e+08 NaN
## 434 9872.01558 3.061692e+08 NaN
## 435 9894.81469 3.075850e+08 NaN
## 436 9917.61380 3.090041e+08 NaN
## 437 9940.41292 3.104265e+08 NaN
## 438 9963.21203 3.118521e+08 NaN
## 439 9986.01114 3.132809e+08 NaN
## 440 10008.81025 3.147131e+08 NaN
## 441 10031.60937 3.161485e+08 NaN
## 442 10054.40848 3.175872e+08 NaN
## 443 10077.20759 3.190291e+08 NaN
## 444 10100.00670 3.204743e+08 NaN
## 445 10122.80581 3.219228e+08 NaN
## 446 10145.60493 3.233745e+08 NaN
## 447 10168.40404 3.248295e+08 NaN
## 448 10191.20315 3.262878e+08 NaN
## 449 10214.00226 3.277493e+08 NaN
## 450 10236.80137 3.292141e+08 NaN
## 451 10259.60049 3.306822e+08 NaN
## 452 10282.39960 3.321535e+08 NaN
## 453 10305.19871 3.336281e+08 NaN
## 454 10327.99782 3.351060e+08 NaN
## 455 10350.79694 3.365871e+08 NaN
## 456 10373.59605 3.380715e+08 NaN
## 457 10396.39516 3.395591e+08 NaN
## 458 10419.19427 3.410501e+08 NaN
## 459 10441.99338 3.425443e+08 NaN
## 460 10464.79250 3.440417e+08 NaN
## 461 10487.59161 3.455424e+08 NaN
## 462 10510.39072 3.470464e+08 NaN
## 463 10533.18983 3.485537e+08 NaN
## 464 10555.98895 3.500642e+08 NaN
## 465 10578.78806 3.515780e+08 NaN
## 466 10601.58717 3.530951e+08 NaN
## 467 10624.38628 3.546154e+08 NaN
## 468 10647.18539 3.561390e+08 NaN
## 469 10669.98451 3.576658e+08 NaN
## 470 10692.78362 3.591959e+08 NaN
## 471 10715.58273 3.607293e+08 NaN
## 472 10738.38184 3.622660e+08 NaN
## 473 10761.18096 3.638059e+08 NaN
## 474 10783.98007 3.653491e+08 NaN
## 475 10806.77918 3.668955e+08 NaN
## 476 10829.57829 3.684453e+08 NaN
## 477 10852.37740 3.699982e+08 NaN
## 478 10875.17652 3.715545e+08 NaN
## 479 10897.97563 3.731140e+08 NaN
## 480 10920.77474 3.746768e+08 NaN
## 481 10943.57385 3.762428e+08 NaN
## 482 10966.37297 3.778121e+08 NaN
## 483 10989.17208 3.793847e+08 NaN
## 484 11011.97119 3.809606e+08 NaN
## 485 11034.77030 3.825397e+08 NaN
## 486 11057.56941 3.841220e+08 NaN
## 487 11080.36853 3.857077e+08 NaN
## 488 11103.16764 3.872966e+08 NaN
## 489 11125.96675 3.888888e+08 NaN
## 490 11148.76586 3.904842e+08 NaN
## 491 11171.56497 3.920829e+08 NaN
## 492 11194.36409 3.936849e+08 NaN
## 493 11217.16320 3.952901e+08 NaN
## 494 11239.96231 3.968986e+08 NaN
## 495 11262.76142 3.985104e+08 NaN
## 496 11285.56054 4.001254e+08 NaN
## 497 11308.35965 4.017437e+08 NaN
## 498 11331.15876 4.033653e+08 NaN
## 499 11353.95787 4.049901e+08 NaN
## 500 11376.75698 4.066182e+08 NaN
## 501 11399.55610 4.082496e+08 NaN
## 502 11422.35521 4.098842e+08 NaN
## 503 11445.15432 4.115221e+08 NaN
## 504 11467.95343 4.131633e+08 NaN
## 505 11490.75255 4.148077e+08 NaN
## 506 11513.55166 4.164554e+08 NaN
## 507 11536.35077 4.181064e+08 NaN
## 508 11559.14988 4.197606e+08 NaN
## 509 11581.94899 4.214181e+08 NaN
## 510 11604.74811 4.230788e+08 NaN
## 511 11627.54722 4.247429e+08 NaN
## 512 11650.34633 4.264102e+08 NaN
## 513 11673.14544 4.280807e+08 NaN
From the graph, we can see that up to a distance of 2100, SQF locations for white people appear to be clustered and from about 2100 to 2400 they are dispersed.. Lets perform dbsan
i’ll begin with eps of 2100 and adjust if needed.
Upon attempting the clusters at 2100, it seems to pick up all the points, therefore i have adjuster the eps distance to 900 (even though elbow plot shows 500) i order to match the same distance as black people. This will allows a fair comparison between the two.
#first check the coordinate reference system of the this spatial polygon:
st_geometry(police)
## Geometry set for 77 features
## Geometry type: MULTIPOLYGON
## Dimension: XY
## Bounding box: xmin: 563069.7 ymin: 4483098 xmax: 609762.3 ymax: 4529952
## Projected CRS: WGS 84 / UTM zone 18N
## First 5 geometries:
## MULTIPOLYGON (((580787.3 4504805, 580819.3 4504...
## MULTIPOLYGON (((585412.9 4508493, 585408.9 4508...
## MULTIPOLYGON (((584460.3 4510216, 584702 451008...
## MULTIPOLYGON (((590567.2 4502074, 590552.2 4502...
## MULTIPOLYGON (((584652.4 4502815, 584661.5 4502...
#1.extract the points from the 'spatial points (sp)' data frame
sqf_white_sp_extract <- sqf_white_sp%>%
coordinates(.)%>%
as.data.frame()
#2. run the DBSCAN using the extracted sp points:
#i choose clustering of eviction to be at 100 and min number of 6 points
DB_sqf_white <- sqf_white_sp_extract %>%
fpc::dbscan(.,eps = 900, MinPts = 4)
#now plot the results
plot(DB_sqf_white, sqf_white_sp_extract , main = "SQF-White DBSCAN Output", frame = F)
plot(police$geometry, add=T)
#3. lets double check appropriate eps distance with the elbow plot:
sqf_white_sp_extract%>% #takes the point extracted from sp object (1. above)
dbscan::kNNdistplot(.,k=4)
#shows 4000, howveer this seems too large so i'll keep it at 1000 as selected earlier
The outcome of the dbsan confirms that clustering of SQF on white people seems
fOR the same eps distances, SQF clusters of black people are more apparent in the precincts that fall inside Brooklyn, However there is a similarity in clustering of both black and white SQF cases in Manhattan.
Lets explore the spatial autocorrelation of sqf white people to make a comparison.
Spatial weight matrix: Before performing our clustered analysis, we need to generate a spatial weights matrix
#1. calculate the centroids of all polygons in NYC
centroids_police_w <- police_sqf_white%>%
st_centroid()%>%
st_geometry()
## Warning in st_centroid.sf(.): st_centroid assumes attributes are constant over
## geometries of x
plot(centroids_police_w)
#2. generate neighbor lists
#queen's case neighbors : doesn't take centroids. taken adjacent neighbors.
NL_queens_W <- police_sqf_white %>%
poly2nb(., queen=T)
#or k nearest neighbors : take centroids.
NL_KNN_w <-centroids_police_w %>%
knearneigh(., k=4)
NL_KNN_w <- NL_KNN_w %>%
knn2nb()
#now plot them both: Fist take the NL then the centroid of the spatial object
plot(NL_queens_W, st_geometry(centroids_police_w), col="red")
plot(police_sqf_white$geometry, add=T)
plot(NL_KNN_w, st_geometry(centroids_police_w), col="blue")
#add a map underneath
plot(police_sqf_white$geometry, add=T)
#3. create a spatial weights matrix object from these weights
# using row standardization 'w'
SWM_queens_w <- NL_queens_W %>%
nb2listw(., style="W")
SWM_knn <-NL_KNN %>%
nb2listw(., style="W")
final output map 2:
tmap_mode("plot")
## tmap mode set to plotting
#first make sure density col is numeric
police_sqf_white$density_numeric <- as.numeric(police_sqf_white$density)
#1. Local morans density
local_moran_density_w <- police_sqf_white %>%
pull(density_numeric) %>% #column name from police_sqf_black use the density values or count to get local morans
as.vector()%>%
localmoran(., SWM_knn)%>% #spatial weight matrix of knn (generated from before from spatial data)
as_tibble()
slice_head(local_moran_density_w, n=5) #view the values
## # A tibble: 5 x 5
## Ii E.Ii Var.Ii Z.Ii `Pr(z != E(Ii))`
## <localmrn> <localmrn> <localmrn> <localmrn> <localmrn>
## 1 0.4498512 -0.001430091 0.02639028 2.7779558 0.005470206
## 2 1.0910988 -0.013745052 0.25051721 2.2074054 0.027285748
## 3 0.7334375 -0.011358632 0.20752325 1.6349483 0.102059891
## 4 0.7253888 -0.008842327 0.16196131 1.8244297 0.068087143
## 5 0.3616623 -0.007785350 0.14275316 0.9778224 0.328162159
# 2. Copy the I score (column 1) and the z-score sd. (column 4)) back into police_sqf_black data
police_sqf_white <- police_sqf_white %>%
mutate(density_I =as.numeric(local_moran_density_w$Ii))%>% #local_moran_density is tibble above, specific col =Ii
mutate(density_Iz =as.numeric(local_moran_density_w$Z.Ii))
#3.lets map them out:
#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)
#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))
tmap_mode("plot")
## tmap mode set to plotting
#plot interactive map
tm_shape(police_sqf_white) +
tm_layout( title = "Local Moran's I-
SQF:white People")+
tm_layout(frame = FALSE)+
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="Local Moran's I",
legend.hist= TRUE)+
tm_compass() +
tm_scale_bar() +
tm_legend(position=c(1,0.0))+
tm_credits("(c) New York City Police Department", position=c(0.5,0.0))
final output map 3:
#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)
#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))
#map SQF: black people
tmap_mode("plot")
## tmap mode set to plotting
#plot interactive map
tm1 <- tm_shape(police_sqf_black) +
tm_layout( title = "Local Moran's:
Black People")+
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="Local Moran's I SQF:Black People")+
tm_legend(show=FALSE)+
tm_layout(frame=FALSE)
#map SQF: white people
tmap_mode("plot")
## tmap mode set to plotting
#plot interactive map
tm2 <- tm_shape(police_sqf_white) +
tm_layout( title = "Local Moran's:
White People")+
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="Local Moran's I")+
tm_legend(show=FALSE)+
tm_layout(frame=FALSE)
legend1 <- tm_shape(police_sqf_black) +
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="SQF:Black people",
legend.hist= FALSE) +
tm_scale_bar(position=c(0.2,0.04), text.size=0.6)+
tm_compass(north=0, position=c(0,0.1))+
tm_layout(legend.only = TRUE, legend.position=c(0.5,0.25),asp=0.1)+
tm_credits("(c) New York City Police Department", position=c(0.0,0.0))
#map the legend
legend <- tm_shape(police_sqf_white) +
tm_polygons("density_Iz",
style="fixed",
breaks=breaks1,
palette=MoranColours,
midpoint=NA,
title="SQF:White people",
legend.hist= FALSE) +
tm_layout(legend.only = TRUE, legend.position=c(0.5,0.25),asp=0.1)
#plot them side by side
t=tmap_arrange(tm1, tm2, legend1, legend, ncol=2)
t
Comparing the 2 maps we can see that some precincts exhibit similarities in levels of autocorrelation specifically in staten island (1.6-2) and manhattan. In general we can see high levels of spatial autocorrelation in both maps in precincts inside manhattan which could indicate that although theyr are both high, there isnt necessarily racial-profiling happening in manhattan.
The analysis conducted lead us to reach a conclusion that answers the research question and both hypothesis
Ripleys k indicated spatial clustering was present In both cases of black people and white people.
DBSAN was used to observe clustered pattern and local morans to determine hot spots of spatial autocorrelation. The analysis was first done for all SQF on suspect black people and results showed that clustering was most present in precincts across Manhattan, west of Bronx and north of Brooklyn. The clustering performed on SQF of white cases showed that for a similar eps value, there were similarities in clustering in Manhattan, however sqf clustering of black people were higher in Brooklyn which indicates there may be possibilities of racial profiling of black people in Brooklyn
This Lead us to confirm H1 for the first hypothesis:There are noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC
Moreover, by cross comparing Morans i of both black and white sqf cases showed some precincts exhibit similarities in spatial autocorrelation, with highest levels of spatial autocorrelation for both maps in precincts inside manhattan. This could indicate there there may be other factors such internal governance within these precincts that are causing such strong spatial similarities between these precincts.
This confirms H1 in hypothesis 2: H1: There are are similarities in spatial autocorrelation of (SQF) policy between black and white people
Limitations to the analysis - Ripley’s k is sensitive to the boundaries of the study area - Addition spatial autocorrelation was not explored including gerrys and getis -The eps distance outcome of ripleys K could not be used as it was too large and so 900m assumption was set for both morans i cases. -NYPD collected the data so there may have been a confilict of interest
Policy Making -in areas where there are clusters of SQR on black people such as the bronx, policies to ensure that there is no police violence can be be implemented. This may include internal trainin of police forces within these precincts to ensure that racial profiling and racial profiling does not occur.
Further Exploration -clusters over different periods from 2018 compared to 2020 to compare changes after black lives matter. -clusters of socio economic factors that may influence or be linked to SQF racial cases -local regression models for each cluster:localized analysis and therefore custom policy making for precinct that exhibit similarities in spatial pattern